Cross-sensor object-space correspondence analysis method and system using same

ABSTRACT

A cross-sensor object-space correspondence analysis method for detecting at least one object in a space by using cooperation of a plurality of image sensing devices, the method including: the image sensing devices sending raw data or grid code data of multiple frames of sensed images to a main information processing device to determine a corresponding projection point or a moving track of each of the at least one object on a reference plane corresponding to the space, where each of the image sensing devices has an image plane, and the raw data and each of the grid code data all correspond to a time record.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a method for detecting the position ofan object in a space, in particular to a method for locating an objectin a space by using a cross-sensor cooperative detection scheme.

Description of the Related Art

In general buildings or stores, cameras are installed at the corners ofthe internal space, and multiple screens are set up in a monitoring roomfor a security guard to monitor the internal space of the building orthe store, so that the security guard can respond to emergencies in theinternal space in time.

However, general cameras installed in buildings or stores only displaythe captured images or the analysis results of the captured images oncorresponding screens respectively, and do not have a collaborativeprocessing function. Therefore, for the security guard responsible formonitoring the screens, it is not only difficult to stay focused for along time when monitoring multiple screens at the same time, but alsodifficult to identify abnormal events or suspicious persons.

To solve the problems mentioned above, a novel space object detectionscheme is urgently needed.

SUMMARY OF THE INVENTION

One objective of the present invention is to provide a cross-sensorobject-space correspondence analysis method, which can efficientlylocate each object in a space without the need of calculating thetraditional three-dimensional space coordinates by configuring aplurality of grids on an image plane of each image sensing device, andassigning each of the grids with a code.

Another objective of the present invention is to provide a cross-sensorobject-space correspondence analysis method, which can combine theappearance time points of an object with appearance grid codes on theimage planes of the image sensing devices repeatedly to detect atrajectory of the object in a space efficiently.

Still another objective of the present invention is to provide across-sensor object-space correspondence analysis system, which canefficiently execute the object-space correspondence analysis method ofthe present invention by adopting an edge computing architecture.

To achieve the above objectives, a cross-sensor object-spacecorrespondence analysis method is proposed for detecting at least oneobject in a space by using cooperation of a plurality of image sensingdevices, the method being implemented by an edge computing architectureincluding a plurality of information processing units in the imagesensing devices respectively and a main information processing device,and the method including:

the information processing units transmitting detected data to the maininformation processing device, the detected data being raw data ofplural frames of image sensed by the image sensing devices, or at leastone local grid code or at least one global grid code generated by usinga first inference process to process the raw data, and the maininformation processing device using the detected data to determine aprojection point on a reference plane corresponding to the space foreach of the at least one object, where each of the image sensing deviceshas an image plane, the raw data, each of the at least one local gridcode, and each of the at least one global grid code all correspond to atime record, the first inference process includes: performing a targetlocating procedure on the raw data to locate at least one pixel positionin a frame of the image corresponding to at least one of the at leastone object at a time point; and using a first grid code look-up table toperform a first mapping operation on each of the at least one pixelposition to generate the at least one local grid code corresponding toat least one of the grids of one of the image planes, or using a secondgrid code look-up table to perform a second mapping operation on each ofthe at least one pixel position to generate the at least one global gridcode corresponding to at least one of the grids of the reference plane;and

the main information processing device performing a second inferenceprocedure on the raw data of the frames of image provided by theinformation processing units to generate at least one of the at leastone global grid code, and using the global grid code to represent theprojection point; or using a code-code look-up table to perform a thirdmapping operation on the at least one local grid code provided by eachof the information processing units to obtain at least one of the globalgrid codes to represent the projection point of the at least one objecton the reference plane; or using the global grid code provided by theinformation processing units to represent the projection point, wherethe second inference procedure includes: performing an objectpositioning procedure on the raw data to locate at least one pixelposition in a frame of the image corresponding to at least one of the atleast one object; and using a second grid code look-up table to performthe second mapping operation on each of the at least one pixel positionto generate at least one of the global grid codes corresponding to atleast one of the grids of the reference plane.

In one embodiment, the information processing units have at least onehardware acceleration unit.

In one embodiment, the object locating procedure includes using an AImodule to perform an object recognition procedure on the raw data toidentify at least one of the at least one object.

In one embodiment, each of the grids is a polygon.

In one embodiment, the code-code look-up table is determined accordingto a depression angle of each of the image sensing devices.

In possible embodiments, the local grid codes of the first grid codelook-up tables corresponding to any two of the image sensing devices maybe configured to be same or different from each other.

In possible embodiments, the code-code look-up tables corresponding toany two of the image sensing devices may be configured to be same ordifferent from each other.

In one embodiment, the main information processing device furthercombines appearance time points of one of the at least one object withlocated ones of the local grid codes or the global grid codes on theimage planes of the image sensing devices to detect a motion track inthe space.

To achieve the above objective, the present invention further proposes across-sensor object-space correspondence analysis system, which has theedge computing architecture mentioned above to realize the cross-sensorobject-space correspondence analysis method.

In possible embodiments, the main information processing device may be acloud server or a local server or a computer device.

In possible embodiments, the image sensing devices can communicate withthe main information processing device in a wired or wireless manner.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a flowchart of an embodiment of the cross-sensorobject-space correspondence analysis method of the present invention;

FIG. 2 illustrates a system using the method of FIG. 1, where the systemhas an edge computing architecture, and the edge computing architectureincludes a main information processing device and plural image sensingdevices disposed in plural image sensing devices to make the pluralimage sensing devices to cooperatively detect at least one object in aspace;

FIG. 3 illustrates an embodiment that a reference plane representing thespace shown in FIG. 2 is divided into a plurality of first grids eachhaving a polygonal shape; and

FIG. 4 illustrates that the local codes of a plurality of second gridsof the image plane of an image sensing device shown in FIG. 2 are mappedonto a plurality of global codes on the reference plane according to alook-up table, where the look-up table is determined by a depressionangle of the image sensing device.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

To make it easier for our examiner to understand the objective of theinvention, its structure, innovative features, and performance, we usepreferred embodiments together with the accompanying drawings for thedetailed description of the invention.

The principle of the present invention lies in:

(1) Divide a reference plane representing a space into a plurality offirst grids each having a polygonal shape, and assign a global code toeach of the first grids to represent respective positions of the firstgrids, the global codes being mapped onto the first grids with apredetermined sequence, and the present invention can therefore quicklyreflect the position of an object in the space without the need ofcalculating the position coordinates (x, y) on the reference plane;

(2) Install a plurality of image sensing devices in the space, divideeach of the image planes of the image sensing devices into a pluralityof second grids each having a polygonal shape, and assign a local codeor a global code to each of the second grids in a predetermined sequenceto represent locations of the second grids, where when the second gridsof an image plane are represented by the local codes, the local codeshave to be further mapped onto the global codes on the reference planeaccording to a look-up table, and when the second grids of an imageplane are represented by the global codes, the second grids on the imageplane can directly reflect locations via the global codes; and

(3) Let the information transmitted from the image sensing devices to amain information processing device include a time record and at leastone local code (or at least one global code), so that the maininformation processing device can perform a cross-sensor object-spacecorrespondence analysis method to quickly locate and/or track theobjects in the space.

Please refer to FIGS. 1-4, where FIG. 1 illustrates a flowchart of anembodiment of the cross-sensor object-space correspondence analysismethod of the present invention; FIG. 2 illustrates a system using themethod of FIG. 1, where the system has an edge computing architecture,and the edge computing architecture includes a main informationprocessing device and plural image sensing devices disposed in pluralimage sensing devices to make the plural image sensing devices tocooperatively detect at least one object in a space; FIG. 3 illustratesa scenario that a reference plane representing the space shown in FIG. 2is divided into a plurality of first grids each having a polygonalshape; and FIG. 4 illustrates that the local codes of a plurality ofsecond grids of the image plane of an image sensing device shown in FIG.2 are mapped onto a plurality of global codes on the reference planeaccording to a look-up table, where the look-up table is determined by adepression angle of the image sensing device.

As shown in FIG. 1, the method includes the following steps: installingan edge computing architecture in a space, the edge computingarchitecture including a main information processing device and pluralinformation processing units in plural image sensing devices in thespace, so that the image sensing devices can cooperatively detect atleast one object in the space (step a); the information processing unitstransmitting detected data to the main information processing device,the detected data being raw data of plural frames of image sensed by theimage sensing devices, or at least one local grid code or at least oneglobal grid code generated by using a first inference process to processthe raw data, and the main information processing device using thedetected data to determine a projection point on a reference planecorresponding to the space for each of the at least one object, whereeach of the image sensing devices has an image plane, the raw data, eachof the at least one local grid code, and each of the at least one globalgrid code all correspond to a time record, the first inference processincludes: performing a target locating procedure on the raw data tolocate at least one pixel position in a frame of the image correspondingto at least one of the at least one object at a time point; and using afirst grid code look-up table to perform a first mapping operation oneach of the at least one pixel position to generate the at least onelocal grid code corresponding to at least one of the grids of one of theimage planes, or using a second grid code look-up table to perform asecond mapping operation on each of the at least one pixel position togenerate the at least one global grid code corresponding to at least oneof the grids of the reference plane (step b); and the main informationprocessing device performing a second inference procedure on the rawdata of the frames of image provided by the information processing unitsto generate at least one of the at least one global grid code, and usingthe global grid code to represent the projection point; or using acode-code look-up table to perform a third mapping operation on the atleast one local grid code provided by each of the information processingunits to obtain at least one of the global grid codes to represent theprojection point of the at least one object on the reference plane; orusing the global grid code provided by the information processing unitsto represent the projection point, where the second inference procedureincludes: performing an object positioning procedure on the raw data tolocate at least one pixel position in a frame of the image correspondingto at least one of the at least one object; and using a second grid codelook-up table to perform the second mapping operation on each of the atleast one pixel position to generate at least one of the global gridcodes corresponding to at least one of the grids of the reference plane(step c).

In step a, the information processing units may have at least onehardware acceleration unit.

In step b, the object positioning procedure includes using an AI moduleto perform an object recognition procedure on the raw data to identifyat least one object, and the grids are each a polygon, for example butnot limited to triangle, quadrilateral, or hexagon, etc.

In step c, the code-code look-up table is determined according to thedepression angle of each of the image sensing devices.

In addition, the local grid codes of the first grid code look-up tablescorresponding to any two of the image sensing devices may be configuredto be same or different from each other. For example, the local gridcodes of one of the first grid code look-up tables can be Arabicnumerals, and the local grid codes of another one of the first grid codelook-up tables can be Arabic numerals or English letters.

In addition, the code-code look-up tables corresponding to any two ofthe image sensing devices may be configured to be same or different fromeach other.

In addition, in light of the above description, the present inventioncan also combine the appearance time points of an object in a space withcorresponding grid codes located on the image planes of the imagesensing devices to efficiently detect a trajectory of the object in thespace.

As shown in FIG. 2, the system of the present invention has an edgecomputing architecture 100, which includes a main information processingdevice 110 and a plurality of image sensing devices 120 arranged in aspace, where the main information processing device 110 can be a cloudserver or a local server or a computer device. Each image sensing device120 has an information processing unit 120 a, and each informationprocessing unit 120 a communicates with the main information processingdevice 110 via a wired or wireless network, so as to perform theaforementioned method to make the image sensing devices cooperativelydetect at least one object.

That is, when in operation, the edge computing architecture 100 willexecute the following steps:

(1) The information processing units 120 a transmit detected data to themain information processing device 110, the detected data being raw dataof plural frames of image sensed by the image sensing devices 120, or atleast one local grid code or at least one global grid code generated byusing a first inference process to process the raw data, and the maininformation processing device 110 uses the detected data to determine aprojection point on a reference plane corresponding to the space foreach of the at least one object. Please refer to FIG. 3, whichillustrates an embodiment that a reference plane representing the spaceshown in FIG. 2 is divided into a plurality of first grids each having apolygonal shape. In addition, each of the image sensing devices 120 hasan image plane, and the raw data, each of the at least one local gridcode, and each of the at least one global grid code all correspond to atime record. The first inference process includes: performing a targetlocating procedure on the raw data to locate at least one pixel positionin a frame of the image corresponding to at least one of the at leastone object at a time point; and using a first grid code look-up table toperform a first mapping operation on each of the at least one pixelposition to generate the at least one local grid code corresponding toat least one of the grids of one of the image planes, or using a secondgrid code look-up table to perform a second mapping operation on each ofthe at least one pixel position to generate the at least one global gridcode corresponding to at least one of the grids of the reference plane.Please refer to FIG. 4, which illustrates that the local codes of aplurality of second grids of the image plane of an image sensing device120 shown in FIG. 2 are mapped onto a plurality of global codes on thereference plane according to a look-up table 121, where the look-uptable 121 is determined by a depression angle θ of the image sensingdevice 120.

(2) The main information processing device 110 performs a secondinference procedure on the raw data of the frames of image provided bythe information processing units 120 a to generate at least one of theat least one global grid code, and using the global grid code torepresent the projection point; or uses a code-code look-up table toperform a third mapping operation on the at least one local grid codeprovided by each of the information processing units 120 a to obtain atleast one of the global grid codes to represent the projection point ofthe at least one object on the reference plane; or uses the global gridcode provided by the information processing units 120 a to represent theprojection point, where the second inference procedure includes:performing an object positioning procedure on the raw data to locate atleast one pixel position in a frame of the image corresponding to atleast one of the at least one object; and using a second grid codelook-up table to perform the second mapping operation on each of the atleast one pixel position to generate at least one of the global gridcodes corresponding to at least one of the grids of the reference plane.

(3) The main information processing device 110 combines the appearancetime points of an object in a space with corresponding grid codeslocated on the image planes of the image sensing devices 120 toefficiently detect a trajectory of the object.

As can be seen from the disclosure above, the present invention has thefollowing advantages:

(1) The cross-sensor object-space correspondence analysis method of thepresent invention can efficiently locate each object in a space withoutthe need of calculating the traditional three-dimensional spacecoordinates by configuring a plurality of grids on an image plane ofeach image sensing device, and assigning each of the grids with a code.

(2) The cross-sensor object-space correspondence analysis method of thepresent invention can combine the appearance time points of an object ina space with corresponding grid codes located on the image planes of theimage sensing devices to efficiently detect a trajectory of the objectin the space.

(3) The cross-sensor object-space correspondence analysis system of thepresent invention can efficiently execute the object-spacecorrespondence analysis method of the present invention by adopting anedge computing architecture.

While the invention has been described by way of example and in terms ofpreferred embodiments, it is to be understood that the invention is notlimited thereto. On the contrary, it is intended to cover variousmodifications and similar arrangements and procedures, and the scope ofthe appended claims therefore should be accorded the broadestinterpretation so as to encompass all such modifications and similararrangements and procedures.

In summation of the above description, the present invention hereinenhances the performance over the conventional structure and furthercomplies with the patent application requirements and is submitted tothe Patent and Trademark Office for review and granting of thecommensurate patent rights.

What is claimed is:
 1. A cross-sensor object-space correspondenceanalysis method for detecting at least one object in a space by usingcooperation of a plurality of image sensing devices, the method beingimplemented by an edge computing architecture including a plurality ofinformation processing units in the image sensing devices respectivelyand a main information processing device, and the method including: theinformation processing units transmitting detected data to the maininformation processing device, the detected data being raw data ofplural frames of image sensed by the image sensing devices, or at leastone local grid code or at least one global grid code generated by usinga first inference process to process the raw data, and the maininformation processing device using the detected data to determine aprojection point on a reference plane corresponding to the space foreach of the at least one object, where each of the image sensing deviceshas an image plane, the raw data, each of the at least one local gridcode, and each of the at least one global grid code all correspond to atime record, the first inference process includes: performing a targetlocating procedure on the raw data to locate at least one pixel positionin a frame of the image corresponding to at least one of the at leastone object at a time point; and using a first grid code look-up table toperform a first mapping operation on each of the at least one pixelposition to generate the at least one local grid code corresponding toat least one of the grids of one of the image planes, or using a secondgrid code look-up table to perform a second mapping operation on each ofthe at least one pixel position to generate the at least one global gridcode corresponding to at least one of the grids of the reference plane;and the main information processing device performing a second inferenceprocedure on the raw data of the frames of image provided by theinformation processing units to generate at least one of the at leastone global grid code, and using the global grid code to represent theprojection point; or using a code-code look-up table to perform a thirdmapping operation on the at least one local grid code provided by eachof the information processing units to obtain at least one of the globalgrid codes to represent the projection point of the at least one objecton the reference plane; or using the global grid code provided by theinformation processing units to represent the projection point, wherethe second inference procedure includes: performing an objectpositioning procedure on the raw data to locate at least one pixelposition in a frame of the image corresponding to at least one of the atleast one object; and using a second grid code look-up table to performthe second mapping operation on each of the at least one pixel positionto generate at least one of the global grid codes corresponding to atleast one of the grids of the reference plane.
 2. The cross-sensorobject-space correspondence analysis method as disclosed in claim 1,wherein the information processing units have at least one hardwareacceleration unit.
 3. The cross-sensor object-space correspondenceanalysis method as disclosed in claim 1, wherein the object locatingprocedure includes using an AI module to perform an object recognitionprocedure on the raw data to identify at least one of the at least oneobject.
 4. The cross-sensor object-space correspondence analysis methodas disclosed in claim 1, wherein each of the grids is a polygon.
 5. Thecross-sensor object-space correspondence analysis method as disclosed inclaim 1, wherein the code-code look-up table is determined according toa depression angle of each of the image sensing devices.
 6. Thecross-sensor object-space correspondence analysis method as disclosed inclaim 1, wherein the local grid codes of the first grid code look-uptables corresponding to any two of the image sensing devices areconfigured to be same or different from each other.
 7. The cross-sensorobject-space correspondence analysis method as disclosed in claim 1,wherein the code-code look-up tables corresponding to any two of theimage sensing devices are configured to be same or different from eachother.
 8. The cross-sensor object-space correspondence analysis methodas disclosed in claim 1, wherein the main information processing devicefurther combines appearance time points of one of the at least oneobject with located ones of the local grid codes or the global gridcodes on the image planes of the image sensing devices to detect amotion track in the space.
 9. A cross-sensor object-space correspondenceanalysis system having the edge computing architecture as disclosed inclaim 1 to realize the cross-sensor object-space correspondence analysismethod as disclosed in claim 1, wherein the main information processingdevice is selected from a group consisting of a cloud server, a localserver, and a computer device.
 10. The cross-sensor object-spacecorrespondence analysis system as disclosed in claim 9, wherein theimage sensing devices communicate with the main information processingdevice in a wired or wireless manner.